A communications system consists of a transmitter 20, a channel 22 connecting the transmitter to a receiver 24, and the receiver itself, as shown in FIG. 1. The transmitter transmits a signal consisting of a sequence of symbols, each symbol being transmitted for the same predefined length of time called a unit interval. Historically there have been only two symbols denoted as 0 and 1. Newer communication standards propose using more symbols, e.g., a new proposal at this time is PAM4 which defines four symbols (0, 1, 2 and 3), any one of which can be transmitted in one-unit interval. Regardless of the number of symbols, each symbol is associated with its own signal level which is either its own electrical voltage or its own optical intensity.
An eye diagram is a standard means of graphically representing all possible unit-interval sized waveforms of a signal by overlaying all of them in a graph just one-unit interval wide. The value of each waveform at any point in the unit interval is determined by the following:                the symbol as sent by the transmitter in the unit interval for that particular waveform,        degraded by the dispersion and other characteristics of the channel,        influenced by the symbols of the N previous unit intervals for some N determined by the characteristics of the channel (this is called inter-symbol interference),        influenced by various sources of jitter (Gaussian, periodic),        influenced by various sources of signal noise (Gaussian, periodic),        influenced by crosstalk (electromagnetic interference due to radiation from other nearby signal carriers), and        influenced by other effects (e.g., duty cycle distortion)        
The horizontal axis of an eye diagram is a time axis. The vertical axis is the signal level and is measured in volts for electrical systems and in watts for optical systems. A sample eye diagram 26 for a PAM4 system is shown in FIG. 2. Note that it has three eyes that separate the four signal levels of PAM4. The white crosses 30 are the sampling points as determined by this method.
The nomenclature “eye diagram” takes its name from the shape of the center, clear area of the diagram and its frequent resemblance to a human eye. This area is clear because normally (in the absence of noise or phase jitter) no waveforms pass through it. The importance of an eye diagram (and in particular of the eye) is that the sampling point can be placed almost anywhere in the eye itself.
A sampling point is the point in the unit interval used by the receiver to determine which signal the waveform in any unit interval represents (e.g., in a two symbol system, whether the waveform in any given unit interval represents 0 or 1.) To determine the correct signal, one sampling point is needed in each eye. For example, the waveform in a four-symbol system may represent a 0, 1, 2 or 3 in any given unit interval, so three sampling points are used, one in each of the three eyes. The method described herein is applicable to any eye in any system with any number of symbols.
A sampling point is defined by its two coordinates.                The horizontal coordinate is time and is referred to as the phase of the sampling point. This is the time from the start of every unit interval when the receiver checks the signal level of the received waveform.        The vertical coordinate is the decision threshold, and is typically measured in volts or watts. Using the 0 and 1 symbols as an example, a receiver identifies a waveform that is above the decision threshold at the time of the sampling phase as a 1 and identifies a waveform that is below the decision threshold at that time as a 0.        
In order to unambiguously recognize 0 and 1, there should be a vertical gap around the sampling point. There should also be a horizontal gap around the sampling point, because over time sampling points in real receiver's drift by a small amount (they actually drift in both time and signal level). It is for these reasons that eye diagrams are constructed, the eye identified and the sampling point placed within the eye.
Because a sampling point can drift (receivers are not perfect devices), it is desirable that the sampling point not be placed close to the edge of an eye. Some region around the nominal sampling point should be clear of waveforms. In fact, this is one of the purposes of mask testing. Not knowing the shape of a mask, we instead surround the sampling point with a rectangle into which a mask can later fit. The only questions are the size and aspect ratio of the rectangle. The aspect ratio will be determined from practical knowledge of receivers and a numeric value. The aspect ratio may be selected to allow a certain amount of drift in the horizontal and vertical directions. In one embodiment, the aspect ratio may be selected so that the rectangle captures the same fraction of drift in both the horizontal and vertical directions. For example, one constant may be used for electrical receivers (e.g., X millivolts per picosecond), while another may be used for optical receivers (e.g., Y microwatts per picosecond).
For a given aspect ratio, however, the rectangle size should be made as large as possible. This invention provides a method to maximize the rectangle for a given aspect ratio.
An eye diagram can have concavities. There are at least two reasons for this but none are relevant to this invention. What is relevant, is that the concavities must be avoided when selecting a sampling point, because they limit the useful area of the eye.